CROSS-ENTROPY LOSS

Rethinking Loss Functions for Fact Verification (2024.eacl-short)

Challenge:	Imbalanced data distributions can cause models to overfit to majority classes and output unreliable (mostly overconfident) predictions.
Approach:	They propose to streamline the model development and deployment using focal loss to address imbalanced data distributions.
Outcome:	The proposed model training with focal loss improves calibration and accuracy compared to standard cross-entropy loss.

Copied to clipboard

Yuta Mukobara, Yutaro Shigeto, Masashi Shimbo

Reasoning

Challenge:	Existing objective functions for fact verification fail to capture heterogeneity among verdict classes . cross-entropy loss treats all misclassification types uniformly, which is problematic .
Approach:	They propose two task-specific objective functions that capture the heterogeneity among verdict classes . they use a dictionary-based objective function to classify Wikipedia sentences into three verdict classes.
Outcome:	The proposed objectives outperform the standard cross-entropy loss objective . the proposed objectives are combined with simple class weighting to overcome imbalance .

Streamlining LLMs: Adaptive Knowledge Distillation for Tailored Language Models (2025.naacl-srw)

Copied to clipboard

Prajvi Saxena, Sabine Janzen, Wolfgang Maass

Advancing Sequential Numerical Prediction in Autoregressive Models (2025.acl-short)

Challenge:	Large language models (LLMs) have transformative potential across industries, e.g., enhancing customer service, revolutionizing medical diagnostics, or identifying crises in news articles.
Approach:	They propose to distill compact, parameter-efficient tailored language models from LLMs for domain-specific tasks with comparable performance.
Outcome:	The proposed framework outperforms knowledge distillation frameworks in the crisis domain, where labeled data is scarce.

Copied to clipboard

Xiang Fei, Jinghui Lu, Qi Sun, Hao Feng, Yanjie Wang, Wei Shi, An-Lan Wang, Jingqun Tang, Can Huang

Challenge:	Autoregressive models are the de facto choice for sequence generation tasks, but standard approaches treat digits as independent tokens and apply cross-entropy loss, overlooking the coherent structure of numerical sequences.
Approach:	They propose a novel approach to entropy loss by extending the Earth Mover’s Distance to preserve ordinal relationships between numerical values and sequence-level to penalize the overall discrepancy between predicted and actual sequences.
Outcome:	Extensive experiments show that NTIL improves numerical prediction and integrates effectively with LLMs/MLLMs.

CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation (2020.acl-main)

Copied to clipboard

Lei Shen, Yang Feng

GTA: Supervised-Guided Reinforcement Learning for Text Classification with Large Language Models (2025.findings-emnlp)

Challenge:	Existing methods for generating emotion-controllable response are inadequate due to content consistency and lack of coherence.
Approach:	They propose a framework that extends the emotion-controllable response generation to a dual task to generate emotional responses and emotional queries alternatively.
Outcome:	The proposed framework outperforms baseline models in coherence, diversity, and relation to emotion factors.

Copied to clipboard

Min Zeng, Jingfei Sun, Xueyou Luo, Shiqi Zhang, Li Xie, Caiquan Liu, Xiaoxin Chen

Using Semantic Similarity as Reward for Reinforcement Learning in Sentence Generation (P19-2)

Challenge:	Reinforcement learning fine-tuning methods suffer from inefficient exploration and slow convergence . supervised fine- tuning methods have limited performance ceiling and less solid theoretical foundation .
Approach:	They propose a Guess-Think-Answer framework that combines supervised and supervised learning in a unified training paradigm.
Outcome:	The proposed framework outperforms both standalone SFT and RL training models on three text classification benchmarks.

Copied to clipboard

Go Yasui, Yoshimasa Tsuruoka, Masaaki Nagata

DisComp: A Two-Stage Prompt Optimization Framework Combining Task-Agnostic and Task-Aware Compression (2025.findings-naacl)

Challenge:	Existing models for sentence generation use cross-entropy loss as the loss function . however, cross-etropy is unable to evaluate sentences as a whole and lacks flexibility . et al., 2018: a novel approach to improve sentence generation models .
Approach:	They propose a method to train a model using estimated semantic similarity between output and reference sentences to alleviate cross-entropy loss problems.
Outcome:	The proposed model improves the BLEU scores from the baseline LSTM NMT model.

Copied to clipboard

Liu Quancai, Haihui Fan, Jinchao Zhang, Lixiangfang Lixiangfang, Lichuanrong Lichuanrong, Bo Li

Responsible & Trustworthy NLP

Challenge:	Extended prompts can lead to substantial computational overhead and increased hardware demands, limiting the scalability and efficiency of large language models.
Approach:	They propose a two-stage prompt compression framework that combines task-agnostic and task-based strategies to efficiently compress prompt length without compromising performance.
Outcome:	The proposed framework outperforms task-agnostic and task-specific compression methods on three benchmark datasets and is up to 6.56 faster at inference compared to the best token-level compression method.

Autoregressive Speech Synthesis without Vector Quantization (2025.acl-long)

Copied to clipboard

Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen M. Meng, Furu Wei

A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks (2024.emnlp-industry)

Challenge:	MELLE is a novel language modeling approach for text-to-speech synthesis that generates continuous tokens from text . authors demonstrate that it reduces the need for vector quantization and improves model robustness .
Approach:	They propose to autoregressively generate continuous mel-spectrogram frames directly from text condition, bypassing vector quantization.
Outcome:	The proposed model achieves superior performance across multiple metrics and is more streamlined.

Copied to clipboard

Tianyi Zhang, Atta Norouzian, Aanchan Mohan, Frederick Ducatelle

Understanding Hard Negatives in Noise Contrastive Estimation (2021.naacl-main)

Challenge:	Virtual assistants often can handle only a limited scope of intents.
Approach:	They propose to combine out-of-scope (OOS) rejection with intent classification . they propose to regularize cross-entropy loss with an in-scope embedding reconstruction loss .
Outcome:	The proposed method achieves a 1-4% improvement in the area under the precision-recall curve for rejecting out-of-sample instances without compromising intent classification performance.

Copied to clipboard

Wenzheng Zhang, Karl Stratos

Challenge:	Existing theoretical results in contrastive learning focus on unconditional negative distributions.
Approach:	They propose a method where highest-scoring incorrect labels are chosen as negatives . they also propose 'hard negative mining' where negatives are used as negative examples .
Outcome:	The proposed approach achieves strong results on the task of zero-shot entity linking.

Semi-Supervised Learning for Video Captioning (2020.findings-emnlp)

Copied to clipboard

Ke Lin, Zhuoxin Gan, Liwei Wang

Multimodality

Challenge:	Existing video captioning algorithms are heavily dependent on supervised training data.
Approach:	They propose to train the video captioning model on labeled and unlabeled data jointly in a semi-supervised learning manner.
Outcome:	The proposed model outperforms state-of-the-art semi-supervised learning approaches on VATEX, MSR-VTT and MSVD datasets.

Towards Efficient and Multifaceted Computer-assisted Pronunciation Training Leveraging Hierarchical Selective State Space Model and Decoupled Cross-entropy Loss (2025.naacl-long)

Copied to clipboard

Fu-An Chao, Berlin Chen

Green & Sustainable NLP

Challenge:	APA and MDD are two of the main tasks of computer-assisted pronunciation training (CAPT) systems.
Approach:	They propose a computer-assisted pronunciation training approach that integrates APA and MDD tasks in parallel.
Outcome:	The proposed approach improves on APA and MDD tasks, and achieves an F1 score of 63.85%.

Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning (2021.acl-short)

Copied to clipboard

Zhiyuan Zeng, Keqing He, Yuanmeng Yan, Zijun Liu, Yanan Wu, Hong Xu, Huixing Jiang, Weiran Xu

Aspect Is Not You Need: No-aspect Differential Sentiment Framework for Aspect-based Sentiment Analysis (2022.naacl-main)

Challenge:	Existing methods of OOD detection only focus on whether a sample is correctly classified . lack of real OOD examples leads to poor prior knowledge about these unknown intents .
Approach:	They propose a supervised contrastive learning objective to minimize intra-class variance . they employ an adversarial augmentation mechanism to obtain pseudo diverse views .
Outcome:	The proposed method minimizes intra-class variance by pulling together in-domain intents belonging to the same class and maximizes inter-class variation by pushing apart samples from different classes.

Copied to clipboard

Jiahao Cao, Rui Liu, Huailiang Peng, Lei Jiang, Xu Bai

Sentiment Analysis

Challenge:	Existing approaches to classify aspects with aspect sentiment bias are hard to find .
Approach:	They propose a no-aspect differential sentiment framework for the ABSA task that eliminates aspect sentiment bias and uses differential sentiment loss instead of cross-entropy loss to better classify the sentiments.
Outcome:	The proposed framework can be combined with almost all traditional ABSA methods.

Exploiting Entity BIO Tag Embeddings and Multi-task Learning for Relation Extraction with Imbalanced Data (P19-1)

Copied to clipboard

Wei Ye, Bo Li, Rui Xie, Zhonghao Sheng, Long Chen, Shikun Zhang

D2U: Distance-to-Uniform Learning for Out-of-Scope Detection (2022.naacl-main)

Challenge:	Existing methods to perform relation extraction are feature-based or kernel-based, but the results of our study show that they can improve the performance of a baseline model with more than 10% absolute increase in F1-score.
Approach:	They propose a multi-task architecture which jointly trains a model to perform relation identification with cross-entropy loss and relation classification with ranking loss.
Outcome:	The proposed model outperforms the state-of-the-art models on ACE 2005 Chinese and English corpus and significantly improves the performance of a baseline model with more than 10% increase in F1-score.

Copied to clipboard

Eyup Yilmaz, Cagri Toraman

Challenge:	Existing methods for out-of-scope (OOS) detection use classifier confidence score, but model cannot infer correctly.
Approach:	They propose a zero-shot post-processing step that exploits the classification confidence score and the shape of the entire output distribution.
Outcome:	The proposed method improves performance when there is no OOS training data and learning procedure when OOS data is available.

Ranking-Constrained Learning with Rationales for Text Classification (2022.findings-acl)

Copied to clipboard

Juanyan Wang, Manali Sharma, Mustafa Bilgic

Exploring Supervised and Unsupervised Rewards in Machine Translation (2021.eacl-main)

Challenge:	Existing approaches to text classification use labels and rationales as ranking constraints.
Approach:	They propose a ranking-constrained loss function that combines cross-entropy loss with ranking losses as rationale constraints to speed up deep learning models with limited training data.
Outcome:	The proposed approach outperforms baselines on three human-annotated datasets and shows that it is more efficient than existing approaches.

Copied to clipboard

Julia Ive, Zixu Wang, Marina Fomicheva, Lucia Specia

Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers (2020.findings-emnlp)

Challenge:	Autoregressive sequence-to-sequence (seq2sequ) neural architectures have become the de facto approach in Machine Translation (MT).
Approach:	They propose to make models less reliant on cross-entropy loss and evaluation metrics . they propose an entropicity-regularised RL method that explores the action space .
Outcome:	The proposed method exploits the action space and unsupervised reward function to balance between exploration and exploitation.

Copied to clipboard

Brian Lester, Daniel Pressel, Amy Hemmeter, Sagnik Ray Choudhury, Srinivas Bangalore

Responsible & Trustworthy NLP

Challenge:	Named entity recognition models use a conditional random field as the final layer . current work eschews prior knowledge of how the span encoding scheme works .
Approach:	They propose to constrain the output to suppress illegal transitions to train a tagger with a cross-entropy loss twice as fast as a CRF.
Outcome:	The proposed model trains twice as fast as a CRF with statistically insignificant differences in F1 . the proposed model is open source and can be used in PyTorch and TensorFlow.

A Learning-Exploring Method to Generate Diverse Paraphrases with Multi-Objective Deep Reinforcement Learning (2020.coling-main)

Copied to clipboard

Mingtong Liu, Erguang Yang, Deyi Xiong, Yujie Zhang, Yao Meng, Changjian Hu, Jinan Xu, Yufeng Chen

Self-Supervised Learning for Contextualized Extractive Summarization (P19-1)

Challenge:	Paraphrase generation is of great importance for many downstream tasks in natural language processing.
Approach:	They propose a method to generate sentences as learning objectives from the learned data distribution and employ reinforcement learning to combine these new learning objectives for model training.
Outcome:	The proposed method gains significant diversity and improves generation quality over state-of-the-art datasets.

Copied to clipboard

Hong Wang, Xin Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, William Yang Wang

Towards Faithful Dialogues via Focus Learning (2023.acl-long)

Challenge:	Existing models for extractive summarization are usually trained from scratch with a cross-entropy loss . previous work builds an end-to-end system to learn to choose sentences without explicitly modeling document context .
Approach:	They propose three auxiliary pre-training tasks that learn to capture the document context in a self-supervised fashion.
Outcome:	The proposed models outperform existing models on a CNN/DM dataset.

Copied to clipboard

Yifan Deng, Xingsheng Zhang, Heyan Huang, Yue Hu

Dialogue Systems & Conversational Agents

Challenge:	Existing knowledge-grounded models rely on elaborate data engineering or increasing the model’s parameters ignoring to track the tokens that significantly influence losses, which is decisive for the optimization direction of the model in each iteration.
Approach:	They propose a novel learning approach that adjusts the contribution of each token to the optimization direction by directly scaling the corresponding objective loss.
Outcome:	The proposed approach achieves the new state-of-the-art results and generates more reliable responses while maintaining training stability.

Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales (2024.findings-naacl)

Copied to clipboard

Lucas Resck, Marcos M. Raimundo, Jorge Poco

Explainability & Interpretability in NLP

Challenge:	Saliency post-hoc explainability methods are important tools for understanding complex NLP models, but they may not align with human intuition, making the explanations not plausible.
Approach:	They propose a method for incorporating rationales into text classification models by augmenting the standard cross-entropy loss with a novel loss function inspired by contrastive learning.
Outcome:	The proposed approach enhances the plausibility of post-hoc explanations while preserving their faithfulness.

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport (2023.findings-acl)

Copied to clipboard

Weijie Xu, Xiaoyu Jiang, Srinivasan Sengamedu Hanumantha Rao, Francis Iannacci, Jinjin Zhao

Self-Supervised Curriculum Learning for Spelling Error Correction (2021.emnlp-main)

Challenge:	Recent Neural Topic Models (NTMs) have limited applications in the real world due to the challenge of incorporating human knowledge.
Approach:	They propose a semi-supervised neural topic modeling method, vONTSS, which uses von Mises-Fisher variational autoencoders and optimal transport.
Outcome:	The proposed method outperforms existing semi-supervised topic modeling methods on multiple aspects.

Copied to clipboard

Zifa Gan, Hongfei Xu, Hongying Zan

Multi-Granularity Optimization for Non-Autoregressive Translation (2022.emnlp-main)

Challenge:	Current approaches to SEC typically leverage a pre-training then fine-tuning procedure that treats data equally.
Approach:	They propose a self-supervised curriculum learning approach to improve model performance and model learning.
Outcome:	The proposed approach improves the model training and improves CL measurement.

Copied to clipboard

Yafu Li, Leyang Cui, Yongjing Yin, Yue Zhang

Machine Translation

Challenge:	Non-autoregressive machine translation suffers severe performance deterioration due to the naive independence assumption.
Approach:	They propose a method which collects model behaviours on translation segments of various granularities and integrates feedback for backpropagation to reduce latency.
Outcome:	Experiments on four benchmark datasets show that the proposed method outperforms baseline models trained with cross-entropy loss and achieves the best performance on WMT’16 EnRo and highly competitive results on WTM’14 EnDe.

Generating Dialogue Responses from a Semantic Latent Space (2020.emnlp-main)

Copied to clipboard

Wei-Jen Ko, Avik Ray, Yilin Shen, Hongxia Jin

Dialogue Systems & Conversational Agents

Challenge:	Existing models for dialogue generation are unable to integrate information from multiple semantically similar valid responses of a given prompt.
Approach:	They propose to learn the pair relationship between the prompts and responses as a regression task instead of the end-to-end classification on vocabulary.
Outcome:	The proposed model learns the pair relationship between the prompts and responses on a latent space instead of the end-to-end classification on vocabulary.

ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation (2022.coling-1)

Copied to clipboard

Cunxiao Du, Zhaopeng Tu, Longyue Wang, Jing Jiang

Machine Translation

Challenge:	Recent studies have incorporated approaches to improving the standard cross-entropy loss to ameliorate the effect of multimodality.
Approach:	They propose a new training oaxe loss which removes the penalty of word order errors in the standard cross-entropy loss.
Outcome:	Extensive experiments on NAT benchmarks show that the proposed approach improves translation quality and improves model performance.

Retrieval-Augmented Few-shot Text Classification (2023.findings-emnlp)

Copied to clipboard

Guoxin Yu, Lemao Liu, Haiyun Jiang, Shuming Shi, Xiang Ao

Faster Speculative Decoding via Effective Draft Decoder with Pruned Candidate Tree (2025.acl-long)

Challenge:	Existing methods for retrieval-augmented text classification are successful in the few-shot scenario with limited retrieval space.
Approach:	They propose to use EM-L and R-L to provide task-specific guidance to retrieval metric . they also propose to incorporate retrieved memory alongside parameters for better generalization .
Outcome:	The proposed methods perform better on the few-shot scenario with limited retrieval space.

Copied to clipboard

Huanran Zheng, Xiaoling Wang

Towards Formality-Aware Neural Machine Translation by Leveraging Context Information (2023.findings-emnlp)

Challenge:	Effective Draft Decoder (EDD) is a powerful decoding method that generates more accurate draft tokens by leveraging the encoding results as soft prompts.
Approach:	They propose an effective draft decoder which treats the LLM as a powerful encoder and generates more accurate draft tokens by leveraging the encoding results as soft prompts.
Outcome:	The proposed method significantly improves the performance of large language models and reduces inference latency.

Copied to clipboard

Dohee Kim, Yujin Baek, Soyoung Yang, Jaegul Choo

R-VLM: Region-Aware Vision Language Model for Precise GUI Grounding (2025.findings-acl)

Challenge:	Formality is one of the most important linguistic properties to determine the naturalness of translation.
Approach:	They propose a method to explicitly inform neural machine translation models by pinpointing key informative tokens using a formality classifier.
Outcome:	The proposed method improves translation quality and conforms to the appropriate syntax.

Copied to clipboard

Joonhyung Park, Peng Tang, Sagnik Das, Srikar Appalaraju, Kunwar Yashraj Singh, R. Manmatha, Shabnam Ghadar

Olga Kovaleva, Anna Rumshisky, Alexey Romanov

Challenge:	Existing vision-only GUI agents ground elements from large and cluttered screenshots, requiring them to process substantial irrelevant information that compromises their accuracy.
Approach:	They propose a visual agent model for GUI automation that leverages zoomed-in region proposals for precise element localization.
Outcome:	The proposed approach improves state-of-the-art grounding accuracy by 13% across diverse GUI platforms on the GUI grounding benchmarks ScreenSpot and AgentStudio.

Copied to clipboard

Learning Robust Models for e-Commerce Product Search (2020.acl-main)

Challenge:	a new paper proposes and evaluates a set of loss functions that can be used to train models for representation learning . cross-entropy loss penalizes models when they fail to generate the exact word from ground truth data .
Approach:	They propose and evaluate loss functions that can be used to train any neural model for representation learning.
Outcome:	The proposed loss functions amplify semantic diversity while preserving original meaning . they show performance improvement on paraphrase detection and language inference tasks .

Copied to clipboard

Thanh Nguyen, Nikhil Rao, Karthik Subbian

Beyond Accuracy Optimization: Computer Vision Losses for Large Language Model Fine-Tuning (2024.findings-emnlp)

Challenge:	Existing models that understand search intent are difficult to learn due to lack of labeled datasets.
Approach:	They develop a deep, end-to-end model that learns to effectively classify mismatches . they introduce a latent variable into the cross-entropy loss that alternates between real and generated samples .
Outcome:	The proposed model achieves a relative gain of over 26% in F-score and 17% in Area Under PR curve on live search traffic in multiple countries.

Copied to clipboard

Daniele Rege Cambrin, Giuseppe Gallipoli, Irene Benedetto, Luca Cagliero, Paolo Garza

Multimodality

Challenge:	Large Language Models (LLMs) have demonstrated impressive performance across various tasks, but current training approaches combine cross-entropy loss with extensive data, human feedback, or ad hoc methods to enhance performance.
Approach:	They evaluate the use of established semantic segmentation loss functions in natural language generation to create a versatile, practical, and scalable solution for fine-tuning different architectures.
Outcome:	The proposed models improve on Math Word Problems and question answering tasks without additional data or human feedback.

ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning (2025.findings-emnlp)

Copied to clipboard

Yang Wu, Huayi Zhang, Yizheng Jiao, Lin Ma, Xiaozhong Liu, Jinhong Yu, Dongyu Zhang, Dezhi Yu, Wei Xu

SELFOOD: Self-Supervised Out-Of-Distribution Detection via Learning to Rank (2023.findings-emnlp)

Challenge:	Prevailing methods for task-specific instruction tuning use similarity metrics to select training data . but instruction tuning loss often fails to exhibit a monotonic relationship with actual task performance .
Approach:	They propose a task-specific instruction tuning method that leverages pairwise preference loss as a reward signal.
Outcome:	The proposed method surpasses state-of-the-art methods for task-specific instruction tuning.

Copied to clipboard

Dheeraj Mekala, Adithya Samavedhi, Chengyu Dong, Jingbo Shang

Set Generation Networks for End-to-End Knowledge Base Population (2021.emnlp-main)

Challenge:	Existing methods for supervised OOD detection require expensive manual annotation of in-distribution and OOD samples.
Approach:	They propose a self-supervised OOD detection method that requires only in-distribution samples as supervision.
Outcome:	Experiments with multiple classifiers on coarse- and fine-grained datasets show the proposed method performs well in both coarse-and fine-grid settings.

Copied to clipboard

Dianbo Sui, Chenhao Wang, Yubo Chen, Kang Liu, Jun Zhao, Wei Bi

Information Extraction with Differentiable Beam Search on Graph RNNs (2024.lrec-main)

Challenge:	Existing knowledge base population systems require a machine translation task to generate multiple facts, but the fact order is not considered.
Approach:	They propose a knowledge base population task that aims to discover facts about entities from texts and expand a KB with these facts.
Outcome:	The proposed networks achieve state-of-the-art (SoTA) performance on two benchmark datasets.

Copied to clipboard

Niama El Khbir, Nadi Tomeh, Thierry Charnois

Structured Data in NLP

Challenge:	Existing approaches to information extraction suffer from exposure bias due to discrepancy between training and decoding.
Approach:	They propose to cast graph generation as auto-regressive sequence labeling and make it aware of decoding procedure by using differentiable beam search.
Outcome:	The proposed model outperforms its non-decoding-aware version on ACE05 and ConLL04 datasets.

Improving Long Dialogue Summarization with Semantic Graph Representation (2023.findings-acl)

Copied to clipboard

Yilun Hua, Zhaoyuan Deng, Kathleen McKeown

Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity (2025.acl-long)

Challenge:	Existing algorithms for abstractive summarization of short dialogues are challenging . however, they can generate high-quality summaries for long dialogues .
Approach:	They propose an algorithm that processes complete dialogues into topic-segment-level Abstract Meaning Representation graphs . they propose a pretrained LLM that exploits the text to leverage graph semantics a new text-graph attention .
Outcome:	The proposed algorithm outperforms state-of-the-art models on multiple long dialogue summarization datasets . it also generates additional training signals that facilitate graph feature encoding and content selection.

Copied to clipboard

Yuri Kuratov, Mikhail Arkhipov, Aydar Bulatov, Mikhail Burtsev

CoCo: Coherence-Enhanced Machine-Generated Text Detection Under Low Resource With Contrastive Learning (2023.emnlp-main)

Challenge:	Recent work addresses problem of compression of tokens into shorter sequence of real-valued vectors . attainable lossless compression ratio is typically not higher than x10 .
Approach:	They propose to compress a sequence of tokens into a shorter sequence of real-valued vectors to be used as inputs instead of token embeddings or key-value cache.
Outcome:	The proposed algorithms reduce the amount of compute in existing language models rather than minimizing number of bits needed to store text.

Copied to clipboard

Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, Chao Shen

Multi-domain Hate Speech Detection Using Dual Contrastive Learning and Paralinguistic Features (2024.lrec-main)

Challenge:	Recent proposed methods fail to consider the linguistic structure of texts and lack the ability to handle the low-resource problem.
Approach:	They propose a coherence-based contrastive learning model named CoCo to detect MGTs under low-resource scenario.
Outcome:	The proposed model outperforms state-of-the-art methods on two datasets and two self-constructed datasets.

Copied to clipboard

Somaiyeh Dehghan, Berrin Yanıkoğlu

Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classification (2025.findings-acl)

Challenge:	a recent study shows that hate speech is spread on social networks and can have social and cultural effects . 41% of americans who took the survey have experienced some type of online harassment .
Approach:	They propose a hate speech detection model using contrastive learning loss combined with traditional cross-entropy loss.
Outcome:	The proposed model outperforms comparable models on heated topics from two datasets . the model scored macro-F1 on two- and five-class tasks and averaged for four domains compared .

Copied to clipboard

Subhendu Khatuya, Shashwat Naidu, Saptarshi Ghosh, Pawan Goyal, Niloy Ganguly

Learning Is Not A Race: Improving Retrieval in Language Models via Equal Learning (2025.findings-emnlp)

Challenge:	Existing approaches to multi-label text classification are limited by textual data.
Approach:	They propose a domain-agnostic generative model framework for multi-label text classification that generates predefined label descriptions and matches them to predefined labels.
Outcome:	The proposed model achieves 13.94% and 24.85% performance over all datasets.

Copied to clipboard

Wanqian Yang, Aahlad Manas Puli, Rajesh Ranganath

VCORE: Variance-Controlled Optimization-based Reweighting for Chain-of-Thought Supervision (2026.acl-long)

Challenge:	Overparametrized models trained on cross-entropy loss can overfit on noise . Fitting some tokens early reduces gradient signals in later iterations .
Approach:	They propose to overfit models trained on cross-entropy loss on noise . fitting some tokens early reduces gradient signals in later iterations .
Outcome:	The proposed approaches can be applied to large language models with longer contexts or larger embedding sizes.

Copied to clipboard

Xuan Gong, Senmiao Wang, Hanbo Huang, Ruoyu Sun, Shiyu Liang

Reasoning

Challenge:	Empirical evaluations demonstrate that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models.
Approach:	They propose a framework that reformulates supervision as a constrained optimization problem.
Outcome:	Empirical evaluations show that VCORE achieves the strongest overall average performance, with especially clear gains on lower-capacity models.

Improving Rule-based Reasoning in LLMs using Neurosymbolic Representations (2025.emnlp-main)

Copied to clipboard

Varun Dhanraj, Chris Eliasmith

Reasoning

Challenge:	Large language models (LLMs) face challenges in reliably solving reasoning tasks, especially when solving tasks that require strict rule following.
Approach:	They propose a method that encodes hidden states into neurosymbolic vectors and decodes them into a neurosample vector space to enable problem-solving within a neural space.
Outcome:	The proposed method shows an average of 88.6% lower cross-entropy loss and 15.4 times more problems correctly solved on a suite of mathematical reasoning tasks compared to chain-of-thought prompting and supervised fine-tuning (LoRA).

Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models (2025.acl-long)

Copied to clipboard

Lior Belenki, Alekh Agarwal, Tianze Shi, Kristina Toutanova

CURA: Clinical Uncertainty Risk Alignment for Language Model–Based Risk Prediction (2026.acl-long)

Challenge:	Existing methods to optimize language model pre-training data mixtures are difficult due to the complexity of the data mixture.
Approach:	They propose a method to optimize language model pre-training data mixtures by approximating cross-entropy loss via a Mixture of Data Experts (MDE).
Outcome:	The proposed method improves performance on a slimPajama dataset with a mixture of data experts.

Copied to clipboard

Sizhe Wang, Ziqi Xu, Claire Najjuuko, Charles Alba, Chenyang Lu

Mapping 1,000+ Language Models via the Log-Likelihood Vector (2025.acl-long)

Challenge:	Clinical language models (LMs) are increasingly applied to support clinical risk prediction from free-text notes, yet their uncertainty estimates are poorly calibrated and clinically unreliable.
Approach:	They propose a framework that aligns clinical LM-based risk estimates and uncertainty with individual error likelihoods and cohort-level ambiguities.
Outcome:	The proposed framework improves accuracy on clinical risk prediction tasks without compromising discrimination.

Copied to clipboard

Momose Oyama, Hiroaki Yamagiwa, Yusuke Takase, Hidetoshi Shimodaira